Better annotate interprocess_exception by Dr15Jones · Pull Request #3 · cms-externals/boost

Dr15Jones · 2022-12-15T20:07:55Z

The IBs with the external generator process are periodically failing with the unhelpful error message

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
	26077_0 process: caught exception 
	boost::interprocess_exception::library_error 15-Dec-2022 03:59:39 CET
	  while serialize lumi
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

That message happens when boost::interprocess_exception constructor is passed an error_info that is set to 0 AND the constructor is not passed a const char*. I went and modified all the places our code might reach a function that calls boost::interprocess_exception and forces it to also pass a const char* saying why the exception is being thrown.

Hopefully this will help track down the problems we see ever so often.

cmsbuild · 2022-12-15T20:08:22Z

A new Pull Request was created by @Dr15Jones (Chris Jones) for branch cms/v1.80.0.

@cmsbuild, @smuzaffar, @aandvalenzuela, @iarspider can you please review it and eventually sign? Thanks.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.
cms-bot commands are listed here

Dr15Jones · 2022-12-15T20:08:54Z

@smuzaffar I compiled all CMSSW code that includes boost/interprocess and had the headers redirected to this changed copy of boost.

fwyzard · 2022-12-15T20:16:32Z

boost/interprocess/detail/managed_open_or_create_impl.hpp

               //trying to open, and tried MaxCreateOrOpenTries times. Something fishy
               //is happening here and we can't solve it
-               throw interprocess_exception(error_info(corrupted_error));
+              throw interprocess_exception(error_info(corrupted_error), "file exists when create but not when open");


Suggested change

throw interprocess_exception(error_info(corrupted_error), "file exists when create but not when open");

throw interprocess_exception(error_info(corrupted_error), "do_create_else_open: file exists when trying to create, but not when trying to open");

For consistency with the other cases where you mention the function name explicitly.

fwyzard · 2022-12-15T20:18:01Z

boost/interprocess/managed_shared_memory.hpp

@@ -1,4 +1,4 @@
-//////////////////////////////////////////////////////////////////////////////
+/////////////////////////////////////////////////////////////////////////////


I guess this could be avoided ?

fwyzard · 2022-12-15T20:18:55Z

boost/interprocess/sync/posix/semaphore_wrapper.hpp

      {
         error_info err(other_error);
-         throw interprocess_exception(err);
+         throw interprocess_exception(err, "semaphore_open unknown type");


Suggested change

throw interprocess_exception(err, "semaphore_open unknown type");

throw interprocess_exception(err, "semaphore_open: unknown type");

fwyzard · 2022-12-15T20:19:16Z

boost/interprocess/sync/posix/semaphore_wrapper.hpp

   //Check for error
   if(handle == BOOST_INTERPROCESS_POSIX_SEM_FAILED){
-      throw interprocess_exception(error_info(errno));
+     throw interprocess_exception(error_info(errno),"semaphore_open sem_open failed");


Suggested change

throw interprocess_exception(error_info(errno),"semaphore_open sem_open failed");

throw interprocess_exception(error_info(errno),"semaphore_open: sem_open failed");

fwyzard · 2022-12-15T20:20:06Z

boost/interprocess/sync/posix/semaphore_wrapper.hpp

   if(ret == -1){
      error_info err = system_error_code();
-      throw interprocess_exception(err);
+      throw interprocess_exception(err, "semaphore_unlink sem_init failed");


Suggested change

throw interprocess_exception(err, "semaphore_unlink sem_init failed");

throw interprocess_exception(err, "semaphore_init: sem_init failed");

cmsbuild · 2022-12-15T20:56:04Z

Pull request #3 was updated.

Avoid case where exception message is just "boost::interprocess_exception::library_error"

cmsbuild · 2022-12-15T21:02:49Z

Pull request #3 was updated.

Dr15Jones · 2022-12-15T21:06:11Z

One thing of note is from looking at the code, it looks like the message boost::interprocess_exception::library_error was never supposed to be issued and instead the error_info was supposed to give something useful. So this change is just trying to figure out where error_info is not working like it was intended and then to use the extra info to figure out our problem.

smuzaffar · 2022-12-15T23:03:19Z

test parameters:

full_cmssw = true

smuzaffar · 2022-12-15T23:03:29Z

please test

cmsbuild · 2022-12-16T09:57:14Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c3a0f0/29648/summary.html
COMMIT: fc6a197
CMSSW: CMSSW_13_0_X_2022-12-15-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-externals/boost/3/29648/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c3a0f0/29648/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c3a0f0/29648/git-merge-result

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 157 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3557521
DQMHistoTests: Total failures: 77
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3557422
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 211 log files, 162 edm output root files, 49 DQM output files
TriggerResults: no differences found

smuzaffar · 2022-12-16T23:28:21Z

+externals

cmsbuild · 2022-12-16T23:28:38Z

This pull request is fully signed and it will be integrated in one of the next cms/v1.80.0 IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

cmsbuild added externals-pending orp-pending pending-signatures tests-pending labels Dec 15, 2022

fwyzard reviewed Dec 15, 2022

View reviewed changes

Dr15Jones force-pushed the annotate_interprocess_exception branch from 0dcaab1 to 777347c Compare December 15, 2022 20:55

Better annotate interprocess_exception

fc6a197

Avoid case where exception message is just "boost::interprocess_exception::library_error"

Dr15Jones force-pushed the annotate_interprocess_exception branch from 777347c to fc6a197 Compare December 15, 2022 21:02

cmsbuild added tests-started and removed tests-pending labels Dec 15, 2022

cmsbuild added tests-approved and removed tests-started labels Dec 16, 2022

smuzaffar merged commit d4f566f into cms-externals:cms/v1.80.0 Dec 16, 2022

cmsbuild added externals-approved fully-signed and removed pending-signatures externals-pending labels Dec 16, 2022

smuzaffar mentioned this pull request Dec 16, 2022

[boost] Better annotate interprocess_exception cms-sw/cmsdist#8234

Merged

	throw interprocess_exception(error_info(corrupted_error), "file exists when create but not when open");
	throw interprocess_exception(error_info(corrupted_error), "do_create_else_open: file exists when trying to create, but not when trying to open");

		@@ -1,4 +1,4 @@
		//////////////////////////////////////////////////////////////////////////////
		/////////////////////////////////////////////////////////////////////////////

	throw interprocess_exception(err, "semaphore_open unknown type");
	throw interprocess_exception(err, "semaphore_open: unknown type");

Conversation

Dr15Jones commented Dec 15, 2022

Uh oh!

cmsbuild commented Dec 15, 2022

Uh oh!

Dr15Jones commented Dec 15, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cmsbuild commented Dec 15, 2022

Uh oh!

cmsbuild commented Dec 15, 2022

Uh oh!

Dr15Jones commented Dec 15, 2022

Uh oh!

smuzaffar commented Dec 15, 2022

Uh oh!

smuzaffar commented Dec 15, 2022

Uh oh!

cmsbuild commented Dec 16, 2022

Comparison Summary

Uh oh!

smuzaffar commented Dec 16, 2022

Uh oh!

cmsbuild commented Dec 16, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants